Implement Content-Length Header in the Blocking API#4124
Closed
Lexazan wants to merge 1 commit intooobabooga:mainfrom
Closed
Implement Content-Length Header in the Blocking API#4124Lexazan wants to merge 1 commit intooobabooga:mainfrom
Lexazan wants to merge 1 commit intooobabooga:mainfrom
Conversation
Lexazan
changed the title
|
I just stumbled upon this, in case it’s helpful I have example code for using textgenwebui’s OpenAI extension in streaming mode in Godot here: https://github.com/teddybear082/godot4-ai-npc-example/blob/main/addons/textgenwebuiOpenAI/TextgenwebuiOpenAI.gd. I know it’s not the same as what you are talking about but figured I would pass along as I had a problem with any streaming API service with Godot’s built in httprequest nodes and struggled with it for quite some time. |
Owner
|
The current API will be deprecated and replaced with the OpenAI-compatible one, which now uses FastAPI after #4430. I assume the behavior described in this PR may be correct by default in FastAPI; if not, a new PR would be welcome. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement Content-Length Header in the Blocking API
I have made changes to the API's response behavior. Previously, the API would send headers first, followed by the body when it was generated. However, I have modified it to wait until the body is fully generated before sending both the headers and the body together. Additionally, I have included the Content-Length header in accordance with the recommendations outlined in the HTTP 1.0 protocol specification.
The reason behind this adjustment stems from an issue I encountered while calling the API from the Godot game engine. In the previous setup, the game engine was not waiting for the complete body but instead emitted the data_received event immediately after receiving the headers. There are most likely other API clients exhibiting similar behavior, and this change should improve compatibility with them while also eliminating any head-scratching for those who are implementing it.
Checklist: